Effects of Diagnostic Errors in Pattern Differentiation and Acupuncture Prescription: A Single-Blinded, Interrater Agreement Study

This study compared the interrater agreement for pattern differentiation and acupoints prescription between two groups of human patients simulated with different diagnostic outcomes. Patients were simulated using a dataset about zangfu patterns and separated into groups (n = 30 each) according to the diagnostic outcome determined by a computational model. A questionnaire with 90 patients was delivered to 6 TCM experts (4-year minimal of clinic experience) who were asked to indicate a single pattern (among 73) and 8 acupoints (among 378). Interrater agreement was higher for pattern differentiation than for acupuncture prescription. Interrater agreement on pattern differentiation was slight for both groups with correct (Light's κ = 0.167, 95% CI = [0.108; 0.254]) and incorrect diagnosis (Light's κ = 0.190, 95% CI = [0.120; 0.286]). Interrater agreement on acupuncture prescription was slight for both groups of correct (ι = 0.029, 95% CI = [0.015; 0.057]) and incorrect diagnosis (ι = 0.040, 95% CI = [0.023; 0.058], P = 0.075). Diagnostic performance of raters yielded the following: accuracy = 60.9%, sensitivity = 21.7%, and specificity = 100%. An overall improvement in the interrater agreement and diagnostic accuracy was observed when the data were analyzed using the internal systems instead of the pattern's labels.